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DETAILED ACTION 



1. This action is responsive to communications: application filed on 03/16/2000, EDS filed 
on 12/17/2000. 

2. Claims 1-5, 7-17, 19-29, 31-41, and 43 are pending in the case. Claims 1, 13, 25, and 37 
are independent claims. 



3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 



4. Claims 1-5, 7-10, 13-17, 19-22, 25-29, 31-34, and 38-40 are rejected under 35 U.S.C. 
103(a) as being unpatentable over Covington et al. (hereafter referred to as Covington), US 
5,524,193 published 6/4/1996 in view of Gormish et al. (hereafter referred to as Gormish), 
US 5,337,362 published 8/9/1994 and Russell et al. (hereafter referred to as Russell), US 
5,905,248 published 5/18/1999. 

Regarding independent claim 1, Covington teaches creating a multimedia annotation 
for a document and combining the document with the multimedia annotation representing at least 
one of an audio sound and a video clip to form a multimedia document in col. 2 lines 39-41 . 
What Covington does not teach is that the annotated document may be paper. Gormish does 
teach adding digital data into a paper document in col. 2 lines 8-40 and col. 3 lines 7-25. 



Claim Rejections - 35 USC §103 
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It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to combine Gormish into Covington to create the claimed invention. It would have 
been obvious and desirable to use the ability to place digitally encoded data onto a paper 
document taught by Gormish and use the said ability to implement the annotation method of 
Covington so that the user could have had increased freedom to read the document in either 
paper form or electronic form. 

Covington does not teach that a multimedia document is generated as part of reproducing 
the paper document via a document reproduction system. Russell does teach that a multimedia 
document is generated as part of reproducing the paper document via a document reproduction 
system in fig. 1, 1A, the abstract and col. 3 lines 50-54. Russell integrates embedded hyperlinks 
pointing to internet resources, which could be audio or video clips, into a paper document. It 
would have been obvious to have combined Russell and Gormish into Covington to have created 
the claimed invention. It would have been obvious to have used Russell to have taught one of 
ordinary skill in the art at the time of the invention how to have created a paper instance of the 
multimedia annotated document created by Covington. Gormish would have additionally taught 
one of ordinary skill in the art at the time of the invention how^ encode different kinds of digital 
communications other than URLs which could have then been embedded into the multimedia 
annotated document and then produced a paper instance. 

Regarding independent claim 13, Covington teaches creating a multimedia annotation 
for a document and combining the document with the multimedia annotation representing at least 
one of an audio sound and a video clip to form a multimedia document in col. 2 lines 39-41. 
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What Covington does not teach is that the annotated document may be paper. Gormish does 
teach adding digital data into a paper document in col. 2 lines 8-40 and col. 3 lines 7-25. 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to combine Gormish into Covington to create the claimed invention. It would have 
been obvious and desirable to use the ability to place digitally encoded data onto a paper 
document taught by Gormish and use the said ability to implement the annotation program on 
machine-readable medium of Covington so that the user could have had increased freedom to 
read the document in either paper form or electronic form. 

Covington does not teach that a multimedia document is generated as part of reproducing 

the paper document via a document reproduction system. Russell does teach that a multimedia 

document is generated as part of reproducing the paper document via a document reproduction 

system in fig. 1, 1A, the abstract and col. 3 lines 50-54. Russell integrates embedded hyperlinks 

pointing to internet resources, which could be audio or video clips, into a paper document. It 

would have been obvious to have combined Russell and Gormish into Covington to have created 

the claimed invention. It would have been obvious to have used Russell to have taught one of 

ordinary skill in the art at the time of the invention how to have created a paper instance of the 

multimedia annotated document created by Covington. Gormish would have additionally taught 

4d 

one of ordinary skill in the art at the time of the invention how^ncode different kinds of digital 
communications other than URLs which could have then been embedded into the multimedia 
annotated document and then produced a paper instance. 

Regarding independent claim 25, Covington teaches a computer system comprising a 
data storage device, a processor coupled to the storage device, and inherently comprising a bus in 
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col. 3 line 63 - col. 4 line 20. Covington also teaches creating a multimedia annotation for a 
document and combining the document with the multimedia annotation representing at least one 
of an audio sound and a video clip to form a multimedia document in col. 2 lines 39-41. What 
Covington does not teach is that the annotated document may be paper. Gormish does teach 
adding digital data, which could be annotations, into a paper document in col. 2 lines 8-40 and 
col. 3 lines 7-25. 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to combine Gormish into Covington to create the claimed invention. It would have 
been obvious and desirable to use the ability to place digitally encoded data onto a paper 
document taught by Gormish and use the said ability to implement the annotation computer 
system of Covington so that the user could have had increased freedom to read the document in 
either paper form or electronic form. 

Covington does not teach that a multimedia document is generated as part of reproducing 
the paper document via a document reproduction system. Russell does teach that a multimedia 
document is generated as part of reproducing the paper document via a document reproduction 
system in fig. 1, 1A, the abstract and col. 3 lines 50-54. Russell integrates embedded hyperlinks 
pointing to internet resources, which could be audio or video clips, into a paper document. It 
would have been obvious to have combined Russell and Gormish into Covington to have created 
the claimed invention. It would have been obvious to have used Russell to have taught one of 
ordinary skill in the art at the time of the invention how to have created a paper instance of the 
multimedia annotated document created by Covington. Gormish would have additionally taught 
one of ordinary skill in the art at the time of the invention how encode different kinds of digital 
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communications other than URLs which could have then been embedded into the multimedia 
annotated document and then produced a paper instance. 

Regarding independent claim 37, Covington teaches creating a document to be used 
with a multimedia annotation in col. 2 lines 39-41 and creating a multimedia annotation 
representing at least one of an audio sound and a video clip in col. 2 lines 49-50. Covington also 
discloses storing an image of the document and the multimedia annotation in col. 4 lines 8-11 
and combining the document and the multimedia annotation to form a multimedia document in 
col. 2 lines 52-55. What Covington does not teach is that the annotated document may be paper. 
Gormish does teach adding digital data, which could be annotations, into a paper document in 
col. 2 lines 8-40 and col. 3 lines 7-25. 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to combine Gormish into Covington to create the claimed invention. It would have 
been obvious and desirable to use the ability to place digitally encoded data onto a paper 
document taught by Gormish and use the said ability to implement the annotation computer 
system of Covington so that the user could have had increased freedom to read the document in 
either paper form or electronic form. 

Covington does not teach that a multimedia document is generated as part of reproducing 
the paper document via a document reproduction system. Russell does teach that a multimedia 
document is generated as part of reproducing the paper document via a document reproduction 
system in fig. 1, 1A, the abstract and col. 3 lines 50-54. Russell integrates embedded hyperlinks 
pointing to internet resources, which could be audio or video clips, into a paper document. It 
would have been obvious to have combined Russell and Gormish into Covington to have created 
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the claimed invention. It would have been obvious to have used Russell to have taught one of 
ordinary skill in the art at the time of the invention how to have created a paper instance of the 
multimedia annotated document created by Covington. Gormish would have additionally taught 
one of ordinary skill in the art at the time of the invention how^encode different kinds of digital 
communications other than URLs which could have then been embedded into the multimedia 
annotated document and then produced a paper instance. 

Regarding dependent claims 2, 14, and 26, Covington does not teach a multimedia 
annotation represented as a bar code printed on a multimedia document. Gormish does teach a 
multimedia annotation represented as a bar code printed on a multimedia document in col. 1 lines 
45-66 and also discloses a more advanced form of digital representation on paper in Fig. 2. 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to combine Gormish into Covington to create the invention as claimed. It would have 
been obvious and desirable to use a bar code to represent the multimedia annotation on the 
multimedia document because bar code scanners were well known and inexpensive and would 
have made the invention economically attractive for interpreting the multimedia data. 

Regarding dependent claims 3, 15, and 27, Covington teaches a multimedia annotation 
which encodes an audio sound in the abstract. 

Regarding dependent claims 4, 16, 28, and 40, Covington teaches a location indicator 
associated with the multimedia annotation is placed on the first multimedia document, wherein 
the location indicator indicates where the multimedia annotation can be retrieved and played in 
col. 2 lines 39-55. 
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Regarding dependent claims 5, 17, 29, and 41, Covington does not teach a location 
indicator which is a Uniform Resource Locator in an encrypted form. Russell does teach a 
location indicator which is a Uniform Resource Locator in an encrypted form in fig. 1, 1 A, the 
abstract and col. 3 lines 50-54. 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to combine Russell and Gormish into Covington to have created the claimed 
invention. It would have been obvious and desirable to use a URL link as the location indicator 
because this would make the invention compatible with the internet, which would have been a 
great advantage resulting in extensive compatibility and ease of storing the multimedia files. 

Regarding dependent claims 7, 19, and 31, Covington does not teach that the 
multimedia document is a paper document. Gormish does teach adding digital data into a paper 
document which could be the combined multimedia document in col. 2 lines 8-40 and col. 3 lines 
7-25/ 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to combine Gormish into Covington to create the claimed invention. It would have 
been obvious and desirable to use the paper document taught by Gormish and use said document 
to implement the multimedia document of Covington so that the user could have had increased 
freedom to read the document in either paper form or electronic form. 

Regarding dependent claims 8, 20, and 32, Covington teaches combining the image of 
the document and the multimedia annotation to form a compound multimedia document in col. 2 
lines 53-55. Covington also teaches storing the image of the document and multimedia 
annotation in col. 4 lines 8-11. 
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What Covington does not explicitly teach is generating an image of the paper document, 
the image of the paper document being unconsciously captured during the reproduction of the 
paper document without user intervention. Russell teaches reproducing a paper document from 
which an image also exists in fig. 1, 1 A, the abstract and col. 3 lines 50-54. It would have been 
obvious to one of ordinary skill in the art at the time of the invention to have combined Russell 
and Gormish into Covington to have created the claimed invention. 

It would have been obvious and desirable to one of ordinary skill in the art at the time the 
invention was made to have simultaneously maintained both a paper version of the multimedia 
annotated document and an electronic image version of the multimedia annotated document so 
that the user would have could have had increased freedom to read the document in either paper 
form or electronic form. 

Regarding dependent claims 9, 21, and 33, Covington does not teach a location 
indicator which is a Uniform Resource Locator printed on a multimedia document. Russell does 
teach a location indicator which is a Uniform Resource Locator printed on a multimedia 
document in fig. 1, 1A, the abstract and col. 3 lines 50-54. 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to combine Russell and Gormish into Covington to have created the claimed 
invention. It would have been obvious and desirable to use a URL link as the location indicator 
because this would make the invention compatible with the internet, which would have been a 
great advantage resulting in extensive compatibility and ease of storing the multimedia files. 

Regarding dependent claims 10, 22, and 34, Covington does not teach a location 
indicator represented as a bar code printed on a multimedia document. Gormish does teach a 
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location indicator represented as a bar code printed on a multimedia document in col. 1 lines 45- 
66 and also discloses a more advanced form of digital representation on paper in Fig. 2. Russell 
also teaches a location indicator represented as a bar code printed on a multimedia document in 
fig. 1, 1A, the abstract and col. 3 lines 50-54. 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to have combined Russell and Gormish into Covington to have created the invention 
as claimed. It would have been obvious and desirable to have used a bar code to represent the 
location indicator on the multimedia document because bar code scanners were well known and 
inexpensive and would have made the invention economically attractive for interpreting the 
multimedia data. 

Regarding dependent claim 38, Covington teaches creating a multimedia annotation for 
a document and combining the document with the multimedia annotation to form a multimedia 
document in col. 2 lines 39-41 . What Covington does not teach is that the annotated document 
may be paper. Gormish does teach adding digital data into a paper document in col. 2 lines 8-40 
and col. 3 lines 7-25. 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to combine Gormish into Covington to create the claimed invention. It would have 
been obvious and desirable to use the ability to place digitally encoded data onto a paper 
document taught by Gormish and use the said ability to implement the annotation method of 
Covington so that the user could have had increased freedom to read the document in either 
paper form or electronic form. 
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Regarding dependent claim 39, Covington does not teach a multimedia annotation 
represented as a bar code printed on a multimedia document. Gormish does teach a multimedia 
annotation represented as a bar code printed on a multimedia document in col. 1 lines 45-66 and 
also discloses a more advanced form or digital representation on paper in Fig. 2. 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to combine Gormish into Covington to create the invention as claimed. It would have 
been obvious and desirable to use a bar code to represent the multimedia annotation on the 
multimedia document because bar code scanners were well known and inexpensive and would 
have made the invention economically attractive for interpreting the multimedia data. 

5. Claims 11-12, 23-24, 35-36, and 43 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Covington et al. (hereafter referred to as Covington), US 5,524,193 
published 06/04/1996 in view of Gormish et al. (hereafter referred to as Gormish), US 
5,337,362 published 08/09/1994 and Russell et al. (hereafter referred to as Russell), US 
5,905,248 published 5/18/1999 as applied to claim 8, 20, 32, and 38 above, and further in 
view of Halliday et al. (hereafter referred to as Halliday), US 5,880,740 published 3/9/1999. 

Regarding dependent claims 11, 23, and 35, Covington does not teach sending a 
multimedia document to a recipient by electronic mail. Halliday does teach sending a 
multimedia document to a recipient by electronic mail in col. 8 lines 5-28. 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to have combined Halliday into Covington in view of Gormish and Russell to have 
created the claimed invention. It would have been obvious and desirable to send the electronic 
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multimedia document to a recipient as a part of reproducing the paper document via the 
document reproduction system in the form of an electronic mail attachment so that the nature of 
the multimedia annotations be retained. One of ordinary skill in the art would have known how 
to set up the system to automatically send the electronic multimedia document as an attachment 
based on the paper document reproduction cue. 

Regarding dependent claims 12, 24, and 36, Covington does not teach a recipient 
receiving an image of a paper document and a multimedia annotation in the form of a 
Multipurpose Internet Mail Extension (MIME). Halliday does teach a recipient receiving an 
image of a paper document and a multimedia annotation in the form of a Multipurpose Internet 
Mail Extension (MIME) in col. 8 lines 5-28. 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to have combined Halliday into Covington in view of Gormish and Russell to have 
created the claimed invention. It would have been obvious and desirable to send the electronic 
multimedia document to a recipient in the form of an electronic mail attachment so that the 
nature of the multimedia annotations be retained. One of ordinary skill in the art would have 
known to send multimedia files as attachments in electronic mail. 

Regarding dependent claim 43, Covington does not teach sending the electronic 
multimedia document to a recipient, wherein the recipient receives the electronic multimedia 
document in the form of an attachment to an electronic mail. Halliday does teach sending the 
electronic multimedia document to a recipient, wherein the recipient receives the electronic 
multimedia document in the form of an attachment to an electronic mail in col. 8 lines 5-28. 
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It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to have combined Halliday into Covington in view of Gormish and Russell to have 
created the claimed invention. It would have been obvious and desirable to send the electronic 
multimedia document to a recipient as a part of reproducing the paper document via the 
document reproduction system in the form of an electronic mail attachment so that the nature of 
the multimedia annotations be retained. One of ordinary skill in the art would have known how 
to set up the system to automatically send the electronic multimedia document as an attachment 
based on the paper document reproduction cue. 



Response to Arguments 
6. Applicant's arguments, see page 1 1 lines 7-13, filed 1/23/2004, with respect to the 
rejection of amended claim 37 under 35 U.S.C. 102(b) as being anticipated by Covington et al., 
US 5,524,193 published 06/04/1996 have been fully considered and are persuasive. Therefore, 
the rejection has been withdrawn. However, upon further consideration, a new ground(s) of 
rejection is made under 35 U.S.C. 103(a) as being unpatentable over Covington et al., US 
5,524,193 published 6/4/1996 in view of Gormish et al. , US 5,337,362 published 8/9/1994 and 
Russell et al., US 5,905,248 published 5/18/1999. Russell teaches producing a paper document 
containing bar-coded URLs which may point to various multimedia resources available via the 
internet. The Examiner believes Russell provides the link between Covington and Gormish to 
render the claimed invention obvious. 
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7. Applicant's arguments filed 1/23/2004 have been fully considered but they are not 
persuasive. The Examiner has additionally cited the reference of Russell in view of Applicant's 
amended claims. Regarding Applicant's argument in pages 12-13 that neither Covington nor 
Gormish individually or in combination discloses or suggests the limitations of the independent 
claims 1,13, 25, and 37, the Examiner believes that the combination of Covington, Gormish and 
Russell render the claimed invention obvious. Covington teaches a embedding and linking to 
multimedia annotations within an electronic document, which may be one of at least an audio 
clip or a video clip. Gormish teaches placing digital data and digital communications on a paper 
document. 

To one of ordinary skill in the art at the time of the invention, Russell brings these 
together by teaching creating a paper instance of an electronic document which contains 
embedded barcode links of Universal Resource Locators, which may point to multimedia files 
such as audio clips and video clips. The barcodes embedded throughout the document in Russell 
may be scanned by a computer system and an information transaction is then initiated. The 
information specified by the URL barcode on the paper document is presented to the user on the 
computer system. The Examiner believes the combination of Covington, Gormish, and Russell 
by one of ordinary skill in the art at the time of the invention renders the claimed invention 
obvious. 

Conclusion 

8. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. Cragun et al. ? US 5,804,803 published 9/8/1998 discloses scanning and translating a 
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printed code into a URL that specifies the location of information that is relevant to the object 
and then retrieving and communicating the information to the user. Shih et al., US 6,674,923 Bl 
filed 3/28/2000 discloses locating and accessing digitally stored images from a hard copy print. 

9. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the mailing 
date of this final action. 

10. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Peter J Smith whose telephone number is 703-305-593 1 . The 
examiner can normally be reached on Mondays-Fridays 7:00am-3 :30pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Joseph H Feild can be reached on 703-305-9792. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 



PJS 

March 23, 2004 




